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Abstract — In this paper we describe a system based on 
concept maps to access the level of understanding of a student. 
The main purpose of this project is to improve the use of 
artificial intelligence in assessment of student understanding 
using concept maps of a particular topic. A new system is 
developed based on Question-Tag in which a student answers 
questions on a particular topic and according to the answers 
marked, the concept mapping is done by the system. We 
calculate the probability distribution of the concepts identified 
in the concept map. The evaluation of the student understanding 
is done by analyzing the curve of the graph developed by the 
system and this is compared against the expert concept map. 

Index Terms — Question Tag, Concept Mapping, Probability 
Distribution. 

I. INTRODUCTION 

Concept maps are visual representations or diagrams that 
shows concepts or topics and the relationships between them 
in a hierarchical structure. The root concept consists of 
several subtopics connected to it, thus forming a tree of 
concepts. The root concept is the most general one at the top 
and the bottom most subtopics are the most specific ones at 
the top. We can show relationships between two concepts 
using linking phrases. Usually linking phrases are verbs 
depicting the relationship between the concepts. For eg. 
“Abstract data type includes tree”. Here the linking phrase is 
includes which tells us the relationship between the concepts 
„Abstract data type" and „tree". 

Concept maps are mostly used for constructive learning 
activities such as assessing a student''s understanding of basic 
curricular concepts. With the help of these concept maps, we 
can also give a person, feedback on how well he grasps the 
knowledge of a particular topic. It evaluates student learning 
better than multiple choice questions and is much more 
accurate as well as efficient. 

Concept maps can be constructed by the students by linking 
each concept to another. A student must start from the root 
concept which is the most general and work his way down 
the map. He must complete the tree with as much as nodes 
as possible. This concept map is then compared to another 
concept map, which is created by an expert of that particular 


root concept. The results are then analyzed and feedback 
is provided to the student on his performance. 



Fig: An example concept map 


II. RELATED WORK 

A. Intelligent Knowledge Assessment System: 

It is basically a Web-based application which makes use of 
the concept maps as a tool for knowledge assessment. A 
semantic unit of a concept map is a proposition. Concept 
Maps based tasks can be divided in 

• “fill-in-the map” tasks, where the structure of a CM is given 
to a student and he/she must fill it using the provided set of 
concepts and/or linking phrases, and 

• “construct-the-map” tasks, where a student must decide on 
the structure of a CM by him/herself. 

The system has two goals in the context of the integration 
of technology into the traditional educational process: 

• To promote students" knowledge self- assessment, and 

• To support a teacher in systematic assessment of students" 
knowledge and the improvement of learning courses. 

A teacher defines stages of knowledge assessment and 
creates Concept Maps for all of them by specifying relevant 
concepts and relationships among them in such a way that a 
Concept Map of each stage is nothing else then an extension 
of the previous one. During knowledge assessment a student 
is given a Concept Map based task to solve. The student 
solves the Concept Map based task corresponding to a 
current assessment stage. After a student has submitted 
his/her solution, the system compares a studenf's Concept 
Map with the teacher"s one and generates feedback. 

The system presents questions to the user and generates an 
analysis using the answers provided to these questions. The 
system uses a well-defined structured approach in gathering 
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the required information and performing the required 
analysis. Moreover, this system provides feedback to the 
student as well as the teacher. 

B. The AssessToLearn Framework: 

Since the assessment of student learning is an integral and 
essential part of the processes of teaching and learning, and 
the concept maps are a valuable assessment tool which 
promote to the meaningful learning and have a great 
contribution to the knowledge construction process, a 
coherent and integrated framework in the educational 
assessment process has been proposed, referred to as 
AssessToLearn. The AssessToLearn framework consists of a 
three-step process consisting of principles from 
contemporary theories of learning. Considering concept maps 
as the main tool of the assessment toolbox, the assessment 
items may be implemented by employing various concept 
mapping tasks such as 

• the construction of a concept map from scratch 
(“free construction ’’task), and/or 

• using an available list of concepts (“concept list” 
task), and/or 

• fill some blanks concerning concepts/relationships 
(“partial recall framework” task) (Tsai et al., 2001), 

and/or fill some blanks concerning concepts/relationships by 
using an available list (“partial recognition framework” task) 
(Tsai et al., 2001). 

The three steps of the AssessToLearn framework are as 
follows: 

1. Assessing students prior knowledge-Knowledge 
Activation 

2. Promoting Construction & identifying conceptual 
changes-constructing & enriching knowledge. 

3. Assessing knowledge construction-refining knowledge. 

C. Personalized Assessment System Supporting Adaptation 
and Learning (PASS): 

Some of the key contributions of this system are: 

• Identification of prior knowledge of the student 

• Diagnosis of concepts unknown to the student 

• Identification of the growth in a student's overall 
understanding of the topic. 

D. Artificial Intelligence Based Student Learning Evaluation 
(AISLE): 

The main aim is to design a system which will help to evaluate 
the student's understanding of the particular topic by using 
concept maps. By use of concept maps the probability 
distribution of the concepts can be calculated, identified by 
the student. The evaluation of a studenf's understanding can 
be identified by analyzing the curve of the graph given by 
the calculated probability distribution. This technique makes 
extensive use of XML parsing to perform the required 
evaluation. 

A concept map is a very good tool for teaching new concepts 
because it shows a relation between the already known 
concepts and the new ones. 

The previous systems shown have implemented the 
assessment systems in different ways. The task that proves the 
knowledge of students has been paper based tasks and 
computer based tasks like filling empty or incomplete maps, 
Filling concept maps to the computer using IHMC tools [1] 


etc. Various other methods have been proposed like 
separating cards related to a particular concept from a stack 
of cards having related and unrelated concepts [4]. All of 
these suffer from some common drawbacks: 

• Concept maps are difficult to understand and require 
practice. 

• Students have to spend extra time to learn how to make 
concept maps 

• Some students who are used to rote learning may find it 
intimidating. 

It is for these reasons that a new system is proposed. 

III. IMPLEMENTATION 

A concept is a good tool for teaching a new concept. But for 
evaluating student's understanding of a particular topic, they 
must have [4]: 

• A task that bears evidence of the student's understanding 
and knowledge structure. 

• A format for recording the response of the students. 

• A scoring or grading system through which analysis of 
student's grades will be evaluated accurately and consistently. 

A. Assessment Task and Response Format: 

The previous systems define many methods for assessment. 
This system intends to overcome the drawbacks in the 
previous models. So a Question-tag based system is proposed. 



Fig: Architecture Diagram of proposed system 
The proposed system consists of the following setup: 




Quesliw in teat Concept Map 

Fig: Proposed Evaluation System 
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The assessment system takes input as answers to questions. 
Every questions have one or more tags. These tags are nothing 
but the concept id of the concept which the question covers. If 
a student answers a particular question, then depending on the 
correctness of the answer, the student's grasp on a concept is 
measured.. An optimum number of questions are asked. 
Depending on the tags a concept map is created by the system 
in the back end. 

B. Analysis of Concepts: 

The concepts in the concept map are each given scores 
randomly [1] depending on the level of the concept in the 
concept map. The level of the root node in the concept map is 
level 0. The next level is level 1. The level after level 1 is level 
2. The concepts are all assigned scores according to the 
scoring algorithm [1]. 

And then they are given random scores based on [1], 
obviously the level 2 concepts will be highly scored than the 
levels above because it indicates deeper understanding of a 
particular sub concept. The depth of the hierarchy of the 
concepts indicate the depth of the student's understanding of a 
particular topic. 

Based on the above scores Z-scores of a particular concept 
are calculated based on the formula below [1]: 

Score conccpt — Meanofconceptscores 
concept StandardDeviationscorc 

This is called standardization of scores. The properties of 
standardizing the scores are as follows: 

• Mean of scores is always zero. 

• The standard deviation of scores is always 1. 

• The distribution curve for standardized and 
non- standardized curves is the same. 

The value of the z-score of a concept may be positive, 
negative or zero depending on whether it is above, below or 
equal to the mean of all concept scores. 

To calculate the probability values of each concept present 
in the concept maps, following formula is used [1]: 
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C. Evaluation of Concept Maps: 

The standard probability distribution of the concept maps 
developed by students is compared to the probability 
distributions of the concept maps prepared by the experts and 
both are compared. 
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Fig : Line chart distribution for concept map 
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To evaluate the concept maps the parameters that play an 
important role are as follows: 

1) Height of the curve, which represents the standard 
probability distribution value where the mean of the curve is 
equal to zero. 

2) Concept number, which represents the numeric values 
assigned to each of the concepts in the hierarchy. 

3) Leaning of the curve with the standard curve, which 
represents the depth of the topic or supporting concepts about 
the topic. 


IV. CONCLUSION 

In this paper, we have proposed a new method to evaluate 
studenf's understanding using concept maps. This method 
is based on a Question-Tag based approach. This method 
seeks to eliminate the difficulties on previous 
implementations of concept maps which required the student 
to learn how to draw concept maps. This is not necessary in 
our proposed system because the assessment is done through 
question - answer format which the students are familiar with. 
The evaluation is done through comparing the studenf's 
concept maps with the expert concept map [1]. Thus our 
system eases studenf's effort as well as performs a proper 
evaluation. 
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